< Packaging Scripts for Debian | Russ Allbery > Technical Notes > Debian | Debian on HP Firebird 802 > |
Introduction
Setting Up a Chroot
Building Packages
Updating Chroots
Debugging and Fixing Problems
After years of using pbuilder
with cowbuilder
to build
Debian packages, I set up sbuild
on a btrfs file system as an
experiment. I was very pleasantly surprised and will be using this in the
future for new build setups.
It was a bit hard to find documentation for this configuration. This hopefully collects in one place my personal approach. There is more information in the sbuild documentation on the Debian wiki if you want to see more ways to do this.
This requires a btrfs file system and uses the btrfs snapshot and
subvolume support to make schroot
operations very clean and simple.
btrfs is still fairly experimental and has been known to lose data and
have other serious problems, particularly if you create too many snapshots
or let a btrfs volume completely fill. This may be too bleeding edge for
your taste; if so, you can use other snapshot mechanisms instead. See the
general sbuild
documentation.
Start by installing the packages debootstrap
, eatmydata
,
sbuild
, and schroot
.
Then, create a directory for the chroots and a sub-directory for the
snapshots. I use /srv/chroot
and /srv/chroot/snapshots
.
These need to be on a btrfs file system, so if you're not using btrfs for
your root volume, you'll need to find some free space and create a btrfs
volume mounted at whatever path you pick.
Run sbuild-adduser <username>
as root to grant your regular user
the ability to use sbuild
. You'll then need to either log out and
log back in again (cleanest) or use newgrp
to add the sbuild
group to your current supplemental groups.
sbuild
benefits enormously from a local repository cache, since
(unlike pbuilder
) it doesn't maintain its own on-disk cache and
otherwise will download packages from the Internet on every build. You
therefore will also want to install apt-cacher-ng
, create the
following configuration file in /etc/apt-cacher-ng/localhost.conf
so that it only listens to localhost:
# Restrict apt-cacher-ng to listening on localhost. BindAddress: localhost
and then restart apt-cacher-ng
to pick up the configuration change.
As root, create a btrfs subvolume in the path where you'll install the chroot. The convention for chroot names is <distribution>-<architecture>-sbuild. So, for a sid (unstable) amd64 build chroot, I would run:
btrfs subvolume create /srv/chroot/sid-amd64-sbuild
Then, also as root, create the environment for a target architecture and distribution. For example:
sbuild-createchroot --arch=amd64 --include=debhelper,eatmydata \ --components=main,contrib,non-free sid /srv/chroot/sid-amd64-sbuild \ http://localhost:3142/debian
This will create a chroot for sid (unstable) amd64 builds using the local apt-cacher-ng cache.
I always install debhelper
in every build chroot, even though it's
not part of build-essential
and this means this dependencies isn't
checked during the package build, because (a) Lintian checks for that
dependency anyway, and (b) nearly every Debian package uses debhelper and
installing it and its dependencies would otherwise slow down every build.
This and the below configuration file configures sbuild to use
eatmydata
to suppress file system syncs while doing package builds,
for a substantial speed improvement. I don't care about data corruption
during a power failure because the build environment is transient and
throw-away and is contained in a btrfs subvolume.
This command will install the build environment and then create a
temporary configuration file in /etc/schroot/chroot.d
for this
chroot. I then delete that file and replace it with one that has the
settings I want. I manage these files with Puppet templates to add
appropriate aliases and configuration. Here's an example of the results
of that template for my sid chroot:
[sid-amd64-sbuild] description=Debian sid/amd64 users=<username> root-users=root,<username> profile=sbuild type=btrfs-snapshot aliases=UNRELEASED,unstable,sid btrfs-source-subvolume=/srv/chroot/sid-amd64-sbuild btrfs-snapshot-directory=/srv/chroot/snapshots command-prefix=eatmydata
Obviously, replace <username> with your username. The type
setting
is what tells sbuild to use btrfs snapshots for each build, which are
extremely fast.
The aliases
are matched against distribution names in
debian/changelog
when building a package, so include all of the
names you might use for the target distribution. For example, for a
stable chroot, I would use the code name and the word stable
.
The command-prefix
setting tells schroot
to always use
eatmydata
when running commands in this chroot.
You can use sbuild
directly to build a package, but I always use
another framework. For git-buildpackage
, adding:
[DEFAULT] builder = sbuild
to ~/.gbp.conf
does the trick, or one can use the
--git-builder=sbuild
command-line flag. With dgit
, just run
dgit sbuild
.
sbuild
will always update the chroot before building a package, but
this work will be repeated for every package build and will add to how
long it takes to build packages. It's therefore worthwhile to regularly
update every chroot, allowing the update at the start of the build to be a
no-op in most cases.
I do this with a systemd timer unit as an experiment, which adds a bit of complexity and some weird issues. (For instance, I have to add a delay before doing the update because I haven't been able to figure out how to get a timer unit to trigger after restoring from suspend, but only after the network is on-line again.) The core of the script that timer unit runs is just the following shell command:
for chroot in $(schroot -l --exclude-aliases | grep ^source:); do (sbuild-update -udcar "$chroot" 2>&1) \ | mail -s "sbuild $chroot update" "$MAIL" done
I like getting the output from the updates in email, just to satisfy my
curiosity, but you can make this simpler if you don't care about the
output. The important part is to exclude aliases (so that you don't
update chroots multiple times) and only update the source:
chroots,
which are used as the basis for snapshots.
The great thing about chroots is that they're just files on disk. If
there are issues with the configuration of the chroot (particularly any
problems with configuration files, like sources.list
), you can just
edit them directly in /srv/chroot
as root.
schroot
also provides a mechanism to get a shell in the chroot but
have your changes be preserved (unlike the normal sbuild
use case
that throws away the snapshot when the build is done). To do this, run,
as root:
sbuild-shell source:sid-amd64-sbuild
or whatever the name of the chroot you want a shell in is, with
source:
prepended. You can then do things like clean up
half-installed packages, and your changes will be preserved when you exit
the shell.
If you abort an sbuild
command, you may end up with schroot
sessions and created btrfs snapshots that are no longer in used. You can
find these with schroot -l --all
(look for session chroots). If
this happens, you can run:
schroot --end-session --all-sessions
as root to end those sessions and clean up.
< Packaging Scripts for Debian | Russ Allbery > Technical Notes > Debian | Debian on HP Firebird 802 > |